
SGD%2DQN Careful Quasi%2DNewton Stochastic Gradient Descent.

Sequence Labelling SVMs Trained in One Pass.

Solving multiclass support vector machines with LaRank.

The Huller A Simple and Efficient Online SVM.
Abstract. We propose a novel online kernel classifier algorithm that converges to the Hard Margin SVM solution. The same update rule is used to both add and remove support vectors from the current classifier. Experiments suggest that this algorithm matches the SVM accuracies after a single pass over the training examples. This algorithm is attractive when one seeks a competitive classifier with large datasets and limited computing resources. 1
Fast Kernel Classifiers with Online and Active Learning.
Very high dimensional learning systems become theoretically possible when training examples are  abundant. The computing cost then becomes the limiting factor. Any efficient learning algorithm  should at least take a brief look at each example. But should all examples be given equal attention?  This contribution proposes an empirical answer. We first present an online SVM algorithm  based on this premise. LASVM yields competitive misclassification rates after a single pass over  the training examples, outspeeding state-of-the-art SVM solvers. Then we show how active example  selection can yield faster training, higher accuracies, and simpler models, using only a fraction  of the training example labels.
